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Bayesian  Methods  for  Image  Segmentation 

Mary  Comer,  Charles  A.  Bouman,  Marc  De  Graef,  and  Jeff  P.  Simmons 

This  paper  presents  an  introduction  to  Bayesian  methods  for  image  segmentation,  and  provides  some 
examples  of  the  performance  of  these  methods.  Bayesian  image  segmentation  methods  represent  a  class 
of  statistical  approaches  to  the  problem  of  segmentation.  The  idea  behind  Bayesian  techniques  is  to  use 
statistical  image  models  to  incorporate  prior  information  into  the  segmentation  process.  This  is  typically 
done  by  specifying  a  model  for  the  observed  image  to  be  segmented  and  a  model  for  the  segmentation 
image  itself.  These  image  models  are  then  used  to  create  a  cost  function  which  is  optimized  to  obtain 
the  segmentation  result.  Different  Bayesian  segmentation  schemes  are  distinguished  by  several  aspects: 
the  image  models  used,  the  optimization  criterion  used  to  obtain  the  cost  function,  and  the  computational 
approach  used  to  perform  the  optimization. 

•  Image  models.  Perhaps  the  most  commonly  used  statistical  image  model  is  the  Markov  random  field 
(MRF).  This  model  imposes  a  smoothness  on  the  segmentation,  by  imposing  the  constraint  that  neigh¬ 
boring  pixels  are  likely  to  share  the  same  classification.  An  MRF  is  often  used  to  model  the  segmenta¬ 
tion  image,  and  sometimes  is  also  used  to  model  the  observed  image  [1].  The  observed  image  model 
also  often  incorporates  a  model  for  the  noise  in  the  imaging  system.  The  most  common  model  for  the 
noise  is  Gaussian,  although  other  noise  models  can  be  incorporated. 

•  Estimation  criteria.  There  are  several  popular  optimization  criteria  for  Bayesian  segmentation.  Per¬ 
haps  the  most  common  is  the  maximum  a  posteriori  (MAP)  criterion  [2].  The  MAP  estimate  rep¬ 
resents  the  most  likely  segmentation  given  an  observed  image,  and  it  is  obtained  by  maximizing  a 
function  we  will  refer  to  as  the  posterior  likelihood  function.  Another  criterion  for  Bayesian  seg¬ 
mentation  is  the  maximizer  of  the  posterior  marginals  (MPM)  criterion  [3].  The  MPM  estimate  is 
the  segmentation  that  minimizes  the  expected  number  of  misclassified  pixels.  A  third  Bayesian-type 
criterion  for  segmentation  is  the  minimum  mean-squared  error  (MMSE)  estimate  [1].  The  MMSE 
estimate  is  the  segmentation  that  minimizes  the  mean-squared  error  between  the  estimator  and  the 
true  segmentation.  It  can  be  shown  that  the  MMSE  estimate  represents  the  mean  value  obtained  using 
the  posterior  likelihood  function,  as  opposed  to  the  MAP  estimate,  which  maximizes  the  posterior 
likelihood. 

•  Computational  approaches.  One  of  the  difficulties  that  arises  in  Bayesian  segmentation  is  that  the 
optimization  of  the  cost  function  is  generally  nontrivial.  For  this  reason,  iterative  approaches  that 
approximate  the  exact  solution  are  used.  Historically,  greedy  algorithms  [4]  or  Monte  Carlo-based 
approaches  [2, 3]  have  been  used  for  this  purpose.  While  greedy  algorithms  are  fast,  they  sometimes 
give  poor  results,  whereas  the  Monte  Carlo  approaches  are  slower  but  more  accurate.  More  recently, 
graph  cut  methods  [5]  and  belief  propagation  [6]  have  been  proposed  for  Bayesian  image  segmenta¬ 
tion. 

1  Experimental  Results 

Some  results  showing  the  performance  of  a  particular  Bayesian  segmentation  are  given  in  this  section. 
The  results  are  obtained  using  the  EM/MPM  method  described  in  [7-9].  This  method  uses  an  MRF  model 
for  the  segmentation  image,  a  Gaussian  model  for  the  observed  image,  the  MPM  estimation  criterion,  and  a 
Monte  Carlo-based  approach  to  perform  the  segmentation. 

The  image  on  the  left  in  Figure  1  shows  a  Ti  alloy.  The  Ti  sample  was  forged  in  the  (3  phase  field,  leading 
to  large  a  laths  surrounding  large  areas  of  (3  phase.  On  cooling,  the  (3  phase  decomposed  to  give  a  mixture  of 
fine  a  and  (3.  The  two  different  phases  result  in  two  different  textures  in  the  image;  the  texture  classification 
problem  seeks  to  assign  each  pixel  a  label  according  to  whether  it  lies  in  an  a  or  (3  phase  region.  The  image 
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on  the  right  in  Figure  1  shows  the  result  of  applying  EM/MPM.  The  algorithm  is  sufficiently  robust  to  allow 
a  stack  of  serial  section  images  to  be  segmented  and  reconstructed  into  a  3D  volume,  as  shown  in  Figure  2. 

Figures  3  and  4  show  a  sequence  of  images  representing  the  growth  of  a  silicon  nanowire.  These 
nanowires  are  grown  from  a  liquid  AuSi  alloy  [10],  which  appears  as  the  darkest  region  in  each  image. 
The  slightly  lighter  region  growing  at  the  edge  of  the  AuSi  region  is  a  Si  region  that  is  formed  by  a  nucle- 
ation  process  from  the  AuSi.  The  Si  region  is  the  growing  nanowire,  and  we  wish  to  segment  the  nanowire 
from  the  AuSi  material  and  the  background. 

Figures  3  and  4  show  Bayesian  segmentations  of  two  different  nanowire  sequences.  In  each  figure,  the  top 
row  shows  the  original  sequence,  the  middle  row  shows  a  segmentation  using  a  2D  version  of  the  EM/MPM 
method  applied  to  each  image  in  the  sequence  independently,  and  the  bottom  row  shows  the  result  of  a 
3D  version  of  EM/MPM  that  includes  prior  information  about  the  physics  of  the  nanowire  growth  process 
[9].  It  can  be  seen  that  incorporation  of  the  prior  information  greatly  improves  the  segmentation  results. 
This  example  illustrates  the  general  principle  that  prior  knowledge  incorporated  into  Bayesian  segmentation 
methods  can  significantly  improve  segmentation  performance. 
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Figure  1:  Original  Ti  alloy  image  (left)  and  segmented  image  (right).  Image  data  courtesy  of  Prof.  Hamish  Fraser  of  Ohio  State 
University. 
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Figure  2:  Segmentated  3D  stack  of  Ti  alloy  images. 


Figure  3:  Top:  Original  sequence;  Middle:  2D  segmentation;  Bottom:  3D  segmentation.  Image  data  courtesy  of  Dr.  Eric  Stach  of 
Brookhaven  National  Laboratories. 


Figure  4:  Top:  Original  sequence;  Middle:  2D  segmentation;  Bottom:  3D  segmentation.  Image  data  courtesy  of  Dr.  Eric  Stach  of 
Brookhaven  National  Laboratories. 


Figure  5:  Tomography  figure. 
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